Development
###########

The Genode OS framework is accompanied by a scalable build system and tooling
infrastructure that is designed for the creation of highly modular and
portable systems software.
Understanding the underlying concepts is important for leveraging the full
potential of the framework.
This chapter complements Chapter [Getting started] with the explanation of the
coarse-grained source-tree structure (Section [Source-code repositories]),
the integration
of 3rd-party software (Section [Integration of 3rd-party software]),
the build system (Section [Build system]), and system-integration tools
(Section [System integration and automated testing]).
Furthermore, it describes the project's development process in Section
[Git flow].


Source-code repositories
========================

; XXX figure

As briefly introduced in Section [Source-tree structure], Genode's source tree
is organized in the form of several source-code repositories. This
coarse-grained modularization of the source code has the following benefits:

* Source codes of different concerns remain well separated.
  For example, the platform-specific code for each base
  platform is located in a dedicated _base-<platform>_ repository.

* Different abstraction levels and features of the system can be maintained
  in different source-code repositories.
  Whereby the source code contained in the _os_ repository is free from any
  dependency from 3rd-party software, the components hosted in the _libports_
  repository are free to use foreign code.

* Custom developments and experimental features can be hosted in dedicated
  source-code repositories, which do not interfere with Genode's source
  tree. Such a custom repository can be managed independently from Genode
  using arbitrary revision-control systems.

The build-directory configuration defines the set of repositories to
incorporate into the build process. At build time, the build system overlays
the directory structures of all selected repositories
to form a single logical source tree. The selection of source-code
repositories ultimately defines the view of the build system on the source
tree.

Note that the order of the repositories as configured in the build
configuration (in _etc/build.conf_) is important. Front-most repositories
shadow subsequent repositories.
This makes the repository mechanism a powerful tool for tweaking
existing repositories: By adding a custom repository in front of another one,
customized versions of single files (e.g., header files or target description
files) can be supplied to the build system without changing the original
repository.

Each source-code repository has the principle structure shown in Table
[repository_overview].

  Directory     | Description
 ------------------------------------------------------------
  _doc/_        | Documentation, specific for the repository
 ------------------------------------------------------------
  _etc/_        | Default configuration for the build system
 ------------------------------------------------------------
  _mk/_         | Build-system supplements
 ------------------------------------------------------------
  _include/_    | Globally visible header files
 ------------------------------------------------------------
  _src/_        | Source codes and target build descriptions
 ------------------------------------------------------------
  _lib/mk/_     | Library build descriptions
 ------------------------------------------------------------
  _lib/import/_ | Library import descriptions
 ------------------------------------------------------------
  _ports/_      | Port descriptions of 3rd-party software
 ------------------------------------------------------------
  _recipes/_    | Package descriptions for depot content
 ------------------------------------------------------------
  _run/_        | System scenarios in the form of run scripts

[table repository_overview]
  Structure of a source-code repository. Depending on the repository,
  only a subset of those directories may be present.


Integration of 3rd-party software
=================================

Downloaded 3rd-party source code resides outside of the actual repository at
the central _<genode-dir>/contrib/_ directory. This structure has the
following benefits over hosting 3rd-party source code along with Genode's
genuine source code:

* Working with grep within the repositories works very efficient because
  downloaded and extracted 3rd-party code is not in the way. Such code
  resides next to the repositories.

* Storing all build directories and downloaded 3rd-party source code somewhere
  outside the Genode source tree, e.g., on different disk partitions, can
  be easily accomplished by creating symbolic links for the _build/_
  and _contrib/_ directories.

The _contrib/_ directory is managed using the tools at
_<genode-dir>/tool/ports/_.

:Obtain a list of available ports:
  ! tool/ports/list

:Download and install a port:
  ! tool/ports/prepare_port <port-name>

The _prepare_port_ tool scans all repositories under _repos/_ for the specified
port and installs the port into _contrib/_. Each version
of an installed port resides in a dedicated subdirectory within the _contrib/_
directory. The port-specific directory is called port directory. It is named
_<port-name>-<fingerprint>_. The _<fingerprint>_ uniquely identifies
the version of the port (it is a SHA256 hash of the ingredients of the
port). If two versions of the same port are installed, each of them will
have a different fingerprint. So they end up in different directories.

Within a source-code repository, a port is represented by two files, a
_<port-name>.port_ and a _<port-name>.hash_ file. Both files reside at the
_ports/_ subdirectory of the corresponding repository. The
_<port-name>.port_ file is the port description, which declares the
ingredients of the port, e.g., the archives to download and the patches to apply.
The _<port-name>.hash_ file contains the fingerprint of the corresponding
port description, thereby uniquely identifying a version of the port
as expected by the checked-out Genode version.

For step-by-step instructions on how to add a port using the mechanism,
please refer to the porting guide:

:Genode Porting Guide:

  [https://genode.org/documentation/developer-resources/porting]


Build system
============

Build directories
~~~~~~~~~~~~~~~~~

The build system is supposed to never touch the source tree. The procedure of
building components and integrating them into system scenarios is performed
within a distinct _build directory_. One build directory targets a specific
kernel and hardware platform. Because the source tree is decoupled
from the build directory, one source tree can have many different build
directories associated, each targeted at a different platform.

The recommended way for creating a build directory is the use of the
_create_builddir_ tool located at _<genode-dir>/tool/_.
The tool prints usage information along with a list of supported base
platforms when started without arguments.
For creating a new build directory, one of the listed target platforms must be
specified. By default, the new build directory is created at
_<genode-dir>/build/<platform>/_ where _<platform>_ corresponds to the
specified argument.
Alternatively, the default location can be overridden via the optional
'BUILD_DIR=' argument. For example:

! cd <genode-dir>
! ./tool/create_builddir x86_64 BUILD_DIR=/tmp/build.x86_64

This command creates a new build directory for the 64-bit x86 platform
at _/tmp/build.x86_64/_.
For the basic operations available from within the build directory, please
refer to Section [Using the build system].


Configuration
-------------

Each build directory contains a _Makefile_, which is a symbolic link to
_tool/builddir/build.mk_. The makefile is the front end of the build system
and not supposed to be edited. Besides the makefile, there is an _etc/_
subdirectory that contains the build-directory configuration. For most
platforms, there exists merely a single _build.conf_ file, which defines the
source-code repositories to be incorporated into the build process along
with the parameters for the run tool explained in Section [Run tool].

The selection of source-code repositories is defined by the REPOSITORIES
declaration, which contains a list of directories.
The _etc/build.conf_ file as found in a freshly created build directory is
preconfigured to select the source-code repositories
_base-<platform>_, _base_, _os_, and _demo_.
There are a number of commented-out lines that can be uncommented for
enabling additional repositories.


Cleaning
--------

To remove all but kernel-related generated files, use
! make clean

To remove all generated files, use
! make cleanall

Both 'clean' and 'cleanall' won't remove any files from the _bin/_
subdirectory. This makes the _bin/_ a safe place for files that are
unrelated to the build process, yet are required for the integration stage, e.g.,
binary data.


Controlling the verbosity
-------------------------

To understand the inner workings of the build process in more detail, you can
tell the build system to display each directory change by specifying

! make VERBOSE_DIR=

If you are interested in the arguments that are passed to each invocation of
'make', you can make them visible via

! make VERBOSE_MK=

Furthermore, you can observe each single shell-command invocation by specifying

! make VERBOSE=

Of course, you can combine these verboseness toggles for maximizing the noise.


Target descriptions
~~~~~~~~~~~~~~~~~~~

Each build target is represented by a corresponding _target.mk_ file within
the _src/_ subdirectory of a source-code repository.
This file declares the name of the target, the source codes to be incorporated
into the target, and the libraries the target depends on.
The build system evaluates target descriptions using _make_. Hence, the syntax
corresponds to the syntax of makefiles and the principle functionality
of make is available for _target.mk_ files. For example, it is possible to
define custom rules as done in
Section [Building tools to be executed on the host platform].


Target declarations
-------------------

:'TARGET': is the name of the binary to be created. This is the
  only *mandatory variable* to be defined in each _target.mk_ file.

:'LIBS': is the list of libraries that are used by the target.

:'SRC_CC': contains the list of '.cc' source files. The default search location
  for source codes is the directory where the _target.mk_ file resides.

:'SRC_C': contains the list of '.c' source files.

:'SRC_S': contains the list of assembly '.s' source files.

:'SRC_BIN': contains binary data files to be linked to the target.

:'INC_DIR': is the list of include search locations. Directories should
  always be appended by using '+='.

:'REQUIRES': expresses the requirements that must be satisfied in order to
  build the target. More details about the underlying mechanism is provided
  by Section [Platform specifications].

:'CC_OPT': contains additional compiler options to be used for '.c' as
  well as for '.cc' files.

:'CC_CXX_OPT': contains additional compiler options to be used for the
  C++ compiler only.

:'CC_C_OPT': contains additional compiler options to be used for the
  C compiler only.

:'EXT_OBJECTS': is a list of external objects or libraries. This
  declaration is merely used for interfacing Genode with legacy software
  components.


Specifying search locations
---------------------------

When specifying search locations for header files via the 'INC_DIR' variable or
for source files via 'vpath', the use of relative pathnames is illegal. Instead,
the following variables can be used to reference locations within the
source-code repository where the target resides:

:'REP_DIR': is the base directory of the target's source-code repository.
  Normally, specifying locations relative to the base of the repository is
  rarely used by _target.mk_ files but needed by library descriptions.

:'PRG_DIR': is the directory where the _target.mk_ file resides. This
  variable is always to be used when specifying a relative path.

:'$(call select_from_repositories,path/relative/to/repo)':
  This function returns the absolute path for the given repository-relative
  path by looking at all source-code repositories in their configured order.
  Hereby, it is possible to access files or directories that are outside
  the target's source-code repository.

:'$(call select_from_ports,<port-name>)':
  This function returns the absolute path for the _contrib_ directory of the
  specified _<port-name>_. The contrib directory is located at
  _<genode-dir>/contrib/<port-name>-<fingerprint>_ whereby _<fingerprint>_
  uniquely identifies the version of the port as expected by the current state
  of the Genode source tree.


Custom targets accompanying a library or program
------------------------------------------------

There are cases that call for building custom targets in addition to a regular
library or or program. For example, the executable binary of an application
may be accompanied by generated data files. The creation of such build
artifacts can be expressed by custom make rules. However, a rule is triggered
only if it is a dependency of the build target. This can be achieved by adding
the rule to the 'CUSTOM_TARGET_DEPS' variable. For example,

! CUSTOM_TARGET_DEPS += menu_view_styles.tar
!
! menu_view_styles.tar:
!    $(VERBOSE)cd $(PRG_DIR); tar cf $(PWD)/bin/$@ styles


Library descriptions
~~~~~~~~~~~~~~~~~~~~

In contrast to target descriptions that are scattered across the whole source
tree, library descriptions are located at the central place _lib/mk_. Each
library corresponds to a _<libname>.mk_ file. The base of the description file
is the name of the library. Therefore, no 'TARGET' variable needs to be
defined.
The location of source-code files is usually defined relative to '$(REP_DIR)'.
Library-description files support the following additional declaration:

:'SHARED_LIB = yes': declares that the library should be built as a shared
  object rather than a static library. The resulting object will be called
  _<libname>.lib.so_.


Platform specifications
~~~~~~~~~~~~~~~~~~~~~~~

Building components for different platforms likely implicates that portions of
code are tied to certain aspects of the target platform. For example, target
platforms may differ in the following respects:

* The API of the used kernel,
* The hardware architecture such as x86, ARMv7,
* Certain hardware facilities such as a custom device, or
* Other considerations such as software license requirements.

Each of those aspects may influence the build process in different ways.
The build system provides a generic mechanism to steer the build process
according to such aspects.
Each aspect is represented by a tag called _spec value_.
Any platform targeted by Genode can be characterized by a set of such spec
values.

; XXX figure (spec refinements)

The *developer* of a software component knows the constraints of his
software and thus specifies these requirements in the build-description
file of the component.
The *system integrator* defines the platform the software will be
built for by specifying the targeted platform in the SPECS declaration in the
build directory's _etc/specs.conf_ file.
In addition to the (optional) _etc/specs.conf_
file within the build directory, the build system incorporates all
_etc/specs.conf_ files found in the enabled repositories. For example, when
using the Linux kernel as a platform, the _base-linux/etc/specs.conf_ file is
picked up automatically. The build directory's 'specs.conf' file can still be
used to extend the SPECS declarations, for example to enable special features.

Each _<spec>_ in the SPECS variable instructs the build system to

* Include the make-rules of a corresponding _base/mk/spec/<specname>.mk_
  file. This enables the customization of the build process for each platform.

* Search for _<libname>.mk_ files in the _lib/mk/spec/<specname>/_ subdirectory.
  This way, alternative implementations of one and the same
  library interface can be selected depending on the platform specification.

Before a target or library gets built, the build system checks if the REQUIRES
entries of the build description file are satisfied by entries of the SPECS
variable. The compilation is executed only if each entry in the REQUIRES
variable is present in the SPECS variable as supplied by the build directory
configuration.


Building tools to be executed on the host platform
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Sometimes, software requires custom tools that are used to generate source
code or other ingredients for the build process, for example IDL compilers.
Such tools won't be executed on top of Genode but on the host platform
during the build process. Hence, they must be compiled with the tool chain
installed on the host, not the Genode tool chain.

The build system accommodates the building of such host tools as a side
effect of building a library or a target. Even though it is possible to add
the tool-compilation step to a regular build description file, it is
recommended to introduce a dedicated pseudo library for building such tools.
This way, the rules for building host tools are kept separate from rules that
refer to regular targets. By convention, the pseudo library should be named
_<package>_host_tools_ and the host tools should be built at
_<build-dir>/tool/<package>/_ where _<package>_ refers to the name of the
software package the tool belongs to, e.g., qt5 or mupdf. To build a tool
named _<tool>_, the pseudo library contains a custom make rule like the
following:

! $(BUILD_BASE_DIR)/tool/<package>/<tool>:
!     $(MSG_BUILD)$(notdir $@)
!     $(VERBOSE)mkdir -p $(dir $@)
!     $(VERBOSE)...build commands...

To let the build system trigger the rule, add the custom target to the
'HOST_TOOLS' variable:

! HOST_TOOLS += $(BUILD_BASE_DIR)/tool/<package>/<tool>

Once the pseudo library for building the host tools is in place, it can be
referenced by each target or library that relies on the respective tools via
the 'LIBS' declaration. The tool can be invoked by referring to
'$(BUILD_BASE_DIR)/tool/<package>/tool'.

For an example of using custom host tools, please refer to the mupdf package
found within the libports repository. During the build of the mupdf library,
two custom tools fontdump and cmapdump are invoked. The tools are built via
the _lib/mk/mupdf_host_tools.mk_ library description file. The actual mupdf
library (_lib/mk/mupdf.mk_) has the pseudo library 'mupdf_host_tools' listed
in its 'LIBS' declaration and refers to the tools relative to
'$(BUILD_BASE_DIR)'.


Building 3rd-party software
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The source code of 3rd-party software is managed by the mechanism presented in
Section [Integration of 3rd-party software]. Once prepared, such source codes
resides in a subdirectory of _<genode-dir>/contrib/_.

If the build system encounters a target that incorporates
ported source code (that is, a build-description file that calls the
'select_from_ports' function), it looks up the respective _<port-name>.hash_
file in the
repositories as specified in the build configuration. The fingerprint found in
the hash file is used to construct the path to the port directory under
_contrib/_. If that lookup fails, a meaningful error is printed. Any number of
versions of the same port can be installed at the same time. I.e., when
switching Git branches that use different versions of the same port, the build
system automatically finds the right port version as expected by the currently
active branch.


System integration and automated testing
========================================

Genode's portability across kernels and hardware platforms is one of the prime
features of the framework. However, each kernel or hardware platform requires
different considerations when it comes to system configuration, integration, and
booting. When using a particular kernel, profound knowledge
about the boot concept and the kernel-specific tools is required. To
streamline the testing of system scenarios across the many different supported
kernels and hardware platforms, the framework is equipped with tools that
relieve the system integrator from these peculiarities.


Run tool
~~~~~~~~

The centerpiece of the system-integration infrastructure is the so-called run
tool. Directed by a script (run script), it performs all the steps necessary to
test a system scenario. Those steps are:

# *Building* the components of a scenario
# *Configuration* of the init component
# Assembly of the *boot directory*
# Creation of the *boot image*
# *Powering-on* the test machine
# *Loading* of the boot image
# Capturing the *log output*
# *Validation* of the scenario's behavior
# *Powering-off* the test machine

Each of those steps depends on various parameters such as the
used kernel, the hardware platform used to execute the scenario, the
way the test hardware is connected to the test infrastructure
(e.g., UART, AMT, JTAG, network), the way the test hardware is powered or
reset, or the way of how the scenario is loaded into the test hardware.
To accommodate the variety of combinations of these
parameters, the run tool consists of an extensible library of modules.
The selection and configuration of the modules is expressed in the run-tool
configuration. The following types of modules exist:

:boot-dir modules:
  These modules contain the functionality to populate the boot directory
  and are specific to each kernel. It is mandatory to always include the
  module corresponding to the used kernel.

  _(the available modules are: linux, hw, okl4, fiasco, pistachio, nova,_
  _sel4, foc)_

:image modules:
  These modules are used to wrap up all components used by the run script
  in a specific format and thereby prepare them for execution.
  Depending on the used kernel, different formats can be used. With these
  modules, the creation of ISO and disk images is also handled.

  _(the available modules are: uboot, disk, iso)_

:load modules:
  These modules handle the way the components are transfered to the
  target system. Depending on the used kernel there are various options
  to pass on the components. For example, loading from TFTP or via JTAG is handled
  by the modules of this category.

  _(the available modules are: tftp, jtag, fastboot, ipxe)_

:log modules:
  These modules handle how the output of a currently executed run script
  is captured.

  _(the available modules are: qemu, linux, serial, amt)_

:power_on modules:
  These modules are used for bringing the target system into a defined
  state, e.g., by starting or rebooting the system.

  _(the available modules are: qemu, linux, softreset, amt, netio)_

:power_off modules:
  These modules are used for turning the target system off after the
  execution of a run script.

Each module has the form of a script snippet located under the
_tool/run/<step>/_
directory where _<step>_ is a subdirectory named after the module type.
Further instructions about the use of each module (e.g., additional
configuration arguments) can be found in the form of comments inside the
respective script snippets.
Thanks to this modular structure,
an extension of the tool kit comes down to adding a file at the corresponding
module-type subdirectory. This way, custom work flows (such as tunneling JTAG
over SSH) can be accommodated fairly easily.


Run-tool configuration examples
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To execute a run script, a combination of modules may be used. The combination
is controlled via the RUN_OPT declaration contained in the build directory's
_etc/build.conf_ file.
The following examples illustrate the selection and configuration of different
run modules:

Executing NOVA in Qemu
----------------------

!RUN_OPT = --include boot_dir/nova \
!          --include power_on/qemu --include log/qemu --include image/iso

By including 'boot_dir/nova', the run tool assembles a boot directory equipped
with a boot loader and a boot-loader configuration that is able to bootstrap
the NOVA kernel. The combination of the modules 'power_on/qemu' and 'log/qemu'
prompts the run tool to spawn the Qemu emulator with the generated boot image
and fetch the log output of the emulated machine from its virtual comport.
The specification of 'image/iso' tells the run tool to use a bootable
ISO image as a boot medium as opposed to a disk image.


Executing NOVA on a real x86 machine using AMT
----------------------------------------------

The following example uses Intel's advanced management technology (AMT)
to remotely reset a physical target machine ('power_on/amt')
and capture the serial output over network ('log/amt'). In contrast to the
example above, the system scenario is supplied via TFTP ('load/tftp'). Note
that the example requires a working network-boot setup including a TFTP
server, a DHCP server, and a PXE boot loader.

!RUN_OPT = --include boot_dir/nova \
!          --include power_on/amt \
!                  --power-on-amt-host 10.23.42.13 \
!                  --power-on-amt-password 'foo!' \
!          --include load/tftp \
!                  --load-tftp-base-dir /var/lib/tftpboot \
!                  --load-tftp-offset-dir /x86 \
!          --include log/amt \
!                  --log-amt-host 10.23.42.13 \
!                  --log-amt-password 'foo!'

If the test machine has a comport connection to the machine where the run
tool is executed, the 'log/serial' module may be used instead of 'log/amt':

! --include log/serial --log-serial-cmd 'picocom -b 115200 /dev/ttyUSB0'

Executing base-hw on a Raspberry Pi
-----------------------------------

The following example boots a system scenario based on the base-hw kernel on
a Raspberry Pi that is powered via a network-controllable power plug (netio).
The Raspberry Pi is connected to a JTAG debugger, which is used to load the
system image onto the device.

!RUN_OPT = --include boot_dir/hw \
!          --include power_on/netio \
!                  --power-on-netio-ip 10.23.42.5 \
!                  --power-on-netio-user admin \
!                  --power-on-netio-password secret \
!                  --power-on-netio-port 1 \
!          --include power_off/netio \
!                  --power-off-netio-ip 10.23.42.5 \
!                  --power-off-netio-user admin \
!                  --power-off-netio-password secret \
!                  --power-off-netio-port 1 \
!          --include load/jtag \
!          --load-jtag-debugger \
!              /usr/share/openocd/scripts/interface/flyswatter2.cfg \
!          --load-jtag-board \
!              /usr/share/openocd/scripts/interface/raspberrypi.cfg \
!          --include log/serial \
!                  --log-serial-cmd 'picocom -b 115200 /dev/ttyUSB0'


Meaningful default behaviour
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The 'create_builddir' tool introduced in Section [Using the build system]
equips a freshly created build directory with a meaningful
default configuration that depends on the selected platform and the used
kernel. For example, when creating a build directory for the x86_64 base
platform and building a scenario with 'KERNEL=linux', 'RUN_OPT' is
automatically defined as

! RUN_OPT = --include boot_dir/linux \
!           --include power_on/linux --include log/linux


Run scripts
~~~~~~~~~~~

Using run scripts, complete system scenarios can be described in a
concise and kernel-independent way. As
described in Section [A simple system scenario], a run script can be used
to integrate and test-drive the scenario directly from the build directory.
The best way to get acquainted with the concept is by reviewing the run script
for the hello-world example presented in Section [Defining a system scenario].
It performs the following steps:

# Building the components needed for the system using the 'build' command.
  This command instructs the build system to compile the targets listed in
  the brace block. It has the same effect as manually invoking 'make' with
  the specified argument from within the build directory.

# Creating a new boot directory using the 'create_boot_directory' command.
  The integration of the scenario is performed in a dedicated directory at
  _<build-dir>/var/run/<run-script-name>/_. When the run script is finished,
  this boot directory will contain all components of the final system.

# Installing the configuration for the init component into the boot directory
  using the
  'install_config' command. The argument to this command will be written
  to a file called 'config' within the boot directory. It will eventually
  be loaded as boot module and made available by core's ROM service
  to the init component. The configuration of init is explained in
  Chapter [System configuration].

# Creating a bootable system image using the 'build_boot_image' command.
  This command copies the specified list of files from the _<build-dir>/bin/_
  directory to the boot directory and executes the steps
  needed to transform the content of the boot directory into a bootable
  form.
  Under the hood, the run tool invokes the run-module types _boot_dir_ and
  _boot_image_.
  Depending on the run-tool configuration, this form may be an ISO
  image, a disk image, or a bootable ELF image.

# Executing the system image using the 'run_genode_until' command. Depending
  on the run-tool configuration,
  the system image is executed using an emulator or a physical machine.
  Under the hood, this step invokes the run modules of the types
  _power_on_, _load_, _log_, and _power_off_.
  For most platforms, Qemu is used by default. On Linux,
  the scenario is executed by starting core directly from the boot
  directory. The 'run_genode_until' command takes a regular expression
  as argument. If the log output of the scenario matches the specified
  pattern, the 'run_genode_until' command returns. If specifying 'forever'
  as argument, this command will never return.
  If a regular expression is specified, an additional argument determines
  a timeout in seconds. If the regular expression does not match until
  the timeout is reached, the run script will abort.

After the successful completion of a run script, the run tool prints the
message "Run script execution successful.".

Note that the _hello.run_ script does not contain kernel-specific information.
Therefore it can be executed from the build directory of any base platform
via the command 'make run/hello KERNEL=<kernel> BOARD=<board>'.
When invoking 'make' with an argument of the form 'run/<run-script>', the
build system searches all repositories for a run script with the specified name.
The run script must be located in one of the repositories' _run/_ subdirectories
and have the file extension '.run'.


The run mechanism explained
~~~~~~~~~~~~~~~~~~~~~~~~~~~

The run tool is based on _expect_, which is an extension of the Tcl scripting
language that allows for the scripting of interactive command-line-based
programs.
When the user invokes a run script via _make run/<run-script>_, the build
system invokes
the run tool at _<genode-dir>/tool/run/run_ with the run script and the
content of the 'RUN_OPT' definition as arguments. The
run tool is an expect script that has no other purpose than defining several
commands used by run scripts and including the run modules as specified by the
run-tool configuration.
Whereas _tool/run/run_ provides the generic commands, the run modules under
_tool/run/<module>/_ contain all the peculiarities of the various kernels
and boot strategies.
The run modules thereby document
precisely how the integration and boot concept works
for each kernel platform.

Run modules
-----------

Each module consist of an expect source file located in one of the existing
directories of a category. It is named implicitly by its location and the
name of the source file, e.g. _image/iso_ is the name of the image module
that creates an ISO image.
The source file contains one mandatory function:
! run_<module> { <module-args> }

The function is called if the step is executed by the run tool. If its
execution was successful, it returns true and otherwise false. Certain modules
may also call exit on failure.

A module may have arguments, which are - by convention - prefixed with the name
of the module, e.g., 'power_on/amt' has an argument called
'--power-on-amt-host'. By convention, the modules contain accessor functions
for argument values. For example, the function 'power_on_amt_host' in the run module
_power_on/amt_ returns the value supplied to the argument '--power-on-amt-host'.
Thereby, a run script can access the value of such arguments
in a defined way by calling 'power_on_amt_host'. Also, arguments without a value
are treated similarly. For example, for querying the presence of the argument
'--image-uboot-no-gzip', the run module _run/image/uboot_
provides the corresponding function 'image_uboot_use_no_gzip'.
In addition to these functions, a module may have additional public
functions. Those functions may be used by run scripts or other modules.
To enable a run script or module to query the presence of another module,
the run tool provides the function 'have_include'. For example, the presence of
the _load/tftp_ module can be checked by calling 'have_include' with the
argument '"load/tftp"'.


Using run scripts to implement integration tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Because run scripts are actually expect scripts, the whole arsenal of
language features of the Tcl scripting language is available to them. This
turns run scripts into powerful tools for the automated execution of test
cases. A good example is the run script at _repos/libports/run/lwip.run_,
which tests the lwIP stack by running a simple Genode-based HTTP server on the
test machine. It fetches and validates a HTML page from this server. The run
script makes use of a regular expression as argument to the 'run_genode_until'
command to detect the state when the web server becomes ready, subsequently
executes the 'lynx' shell command to fetch the web site, and employs Tcl's
support for regular expressions to validate the result. The run script works
across all platforms that have network support.
To accommodate a high diversity of platforms, parts of the run script depend
on the _spec_ values as defined for the build directory. The spec values
are probed via the 'have_spec' function. Depending on the probed spec
values, the run script uses the 'append_if' and 'lappend_if' commands
to conditionally assemble the init configuration and the list of boot modules.

To use the run mechanism efficiently, a basic understanding of the Tcl
scripting language is required. Furthermore the functions provided by
_tool/run/run_ and the run modules at _tool/run/_ should be studied.


Automated testing across base platforms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To execute one or multiple test cases on more than one base platform, there
exists a dedicated tool at _tool/autopilot_. Its primary purpose is the
nightly execution of test cases. The tool takes a list of platforms and of
run scripts as arguments and executes each run script on each platform.
A platform is a triplet of CPU architecture, board, and kernel. For example,
the following command instructs autopilot to generate a build directory for
the x86_64 architecture and to execute the _log.run_ script for the kernels
board-kernel combinations NOVA on a PC and seL4 on a PC.

! autopilot -t x86_64-pc-sel4 -t x86_64-pc-nova -r log

The build directory for each architecture is created at
_/tmp/autopilot.<username>/<architecture>_ and the output of each run script is
written to a file called _<architecture>.<board>.<kernel>.<run-script>.log_.
On stderr, autopilot prints the statistics about whether or not each run
script executed successfully on each platform. If at least one run script
failed, autopilot returns a non-zero exit code, which makes it straight
forward to include autopilot into an automated build-and-test environment.


Package management
==================

The established system-integration work flow with Genode is based on the _run_
tool as explained in the previous section. It automates the building,
configuration, integration, and testing of Genode-based systems. Whereas the
run tool succeeds in overcoming the challenges that come with Genode's
diversity of kernels and supported hardware platforms, its scalability is
somewhat limited to appliance-like system scenarios: The result of the
integration process is a system image with a certain feature set. Whenever
requirements change, the system image is replaced with a freshly created image
that takes those requirements into account. In practice, there are two
limitations of this system-integration approach:

First, since the run tool implicitly builds all components required for a
system scenario, the system integrator has to compile all components from
source. For example, if a system includes a component based on Qt5, one needs to
compile the entire Qt5 application framework, which induces significant
overhead to the actual system-integration tasks of composing and configuring
components.

Second, general-purpose systems tend to become too complex and diverse to be
treated as system images. When looking at commodity OSes, each installation
differs with respect to the installed set of applications, user preferences,
used device drivers and system preferences. A system based on the run tool's
work flow would require the user to customize the run script of the system for
each tweak. To stay up to date, the user would need to re-create the
system image from time to time while manually maintaining any customizations.
In practice this is a burden very few end users are willing to endure.

The primary goal of Genode's package management is to overcome these
scalability limitations, in particular:

* Alleviating the need to build everything that goes into system scenarios
  from scratch,
* Facilitating modular system compositions while abstracting from technical
  details,
* On-target system update and system development,
* Assuring the user that system updates are safe to apply by providing the
  ability to easily roll back the system or parts thereof to previous versions,
* Securing the integrity of the deployed software,
* Low friction for existing developers.

The design of Genode's package-management concept is largely influenced by Git
as well as the [https://nixos.org/nix/ - Nix] package manager. In particular
the latter opened our eyes to discover the potential that lies beyond the
package management employed in state-of-the art commodity systems. Even though
we considered adapting Nix for Genode and actually conducted intensive
experiments in this direction, we settled on a custom solution that leverages
Genode's holistic view on all levels of the operating system including the
build system and tooling, source structure, ABI design, framework API, system
configuration, inter-component interaction, and the components itself. Whereby
Nix is designed for being used on top of Linux, Genode's whole-systems view
led us to simplifications that eliminated the needs for Nix' powerful features
like its custom description language.


Nomenclature
~~~~~~~~~~~~

When speaking about "package management", one has to clarify what a "package"
in the context of an operating system represents. Traditionally, a package
is the unit of delivery of a bunch of "dumb" files, usually wrapped up in
a compressed archive. A package may depend on the presence of other
packages. Thereby, a dependency graph is formed. To express how packages fit
with each other, a package is usually accompanied with meta data
(description). Depending on the package manager, package descriptions follow
certain formalisms (e.g., package-description language) and express
more-or-less complex concepts such as versioning schemes or the distinction
between hard and soft dependencies.

Genode's package management does not follow this notion of a "package".
Instead of subsuming all deliverable content under one term, we distinguish
different kinds of content, each in a tailored and simple form. To avoid the
clash of the notions of the common meaning of a "package", we speak of
"archives" as the basic unit of delivery. The following subsections introduce
the different categories.
Archives are named with their version as suffix, appended via a slash. The
suffix is maintained by the author of the archive. The recommended naming
scheme is the use of the release date as version suffix, e.g.,
'report_rom/2017-05-14'.


Raw-data archive
----------------

A raw-data archive contains arbitrary data that is - in contrast to executable
binaries - independent from the processor architecture. Examples are
configuration data, game assets, images, or fonts. The content of raw-data
archives is expected to be consumed by components at runtime. It is not
relevant for the build process of executable binaries. Each raw-data
archive contains merely a collection of data files. There is no meta data.


API archive
-----------

An API archive has the structure of a Genode source-code repository. It may
contain all the typical content of such a source-code repository such as header
files (in the _include/_ subdirectory), source codes (in the _src/_
subdirectory), library-description files (in the _lib/mk/_ subdirectory), or
ABI symbols (_lib/symbols/_ subdirectory). At the top level, a LICENSE file is
expected that clarifies the license of the contained source code. There is no
meta data contained in an API archive.

An API archive is meant to provide _ingredients_ for building components. The
canonical example is the public programming interface of a library (header
files) and the library's binary interface in the form of an ABI-symbols file.
One API archive may contain the interfaces of multiple libraries. For example,
the interfaces of libc and libm may be contained in a single "libc" API
archive because they are closely related to each other. Conversely, an API
archive may contain a single header file only. The granularity of those
archives may vary. But they have in common that they are used at build time
only, not at runtime.


Source archive
--------------

Like an API archive, a source archive has the structure of a Genode
source-tree repository and is expected to contain all the typical content of
such a source repository along with a LICENSE file. But unlike an API archive,
it contains descriptions of actual build targets in the form of Genode's usual
'target.mk' files.

In addition to the source code, a source archive contains a file
called 'used_apis', which contains a list of API-archive names with each
name on a separate line. For example, the 'used_apis' file of the 'report_rom'
source archive looks as follows:

! base/2017-05-14
! os/2017-05-13
! report_session/2017-05-13

The 'used_apis' file declares the APIs needed to incorporate into the build
process when building the source archive. Hence, they represent _build-time_
_dependencies_ on the specific API versions.

A source archive may be equipped with a top-level file called 'api' containing
the name of exactly one API archive. If present, it declares that the source
archive _implements_ the specified API. For example, the 'libc/2017-05-14'
source archive contains the actual source code of the libc and libm as well as
an 'api' file with the content 'libc/2017-04-13'. The latter refers to the API
implemented by this version of the libc source package (note the differing
versions of the API and source archives)


Binary archive
--------------

A binary archive contains the build result of the equally-named source archive
when built for a particular architecture. That is, all files that would appear
in the _<build-dir>/bin/_ subdirectory when building all targets present in
the source archive. There is no meta data present in a binary archive.

A binary archive is created out of the content of its corresponding source
archive and all API archives listed in the source archive's 'used_apis' file.
Note that since a binary archive depends on only one source archive, which
has no further dependencies, all binary archives can be built independently
from each other.
For example, a libc-using application needs the source code of the
application as well as the libc's API archive (the libc's header file and
ABI) but it does not need the actual libc library to be present.


Package archive
---------------

A package archive contains an 'archives' file with a list of archive names
that belong together at runtime. Each listed archive appears on a separate line.
For example, the 'archives' file of the package archive for the window
manager 'wm/2018-02-26' looks as follows:

! genodelabs/raw/wm/2018-02-14
! genodelabs/src/wm/2018-02-26
! genodelabs/src/report_rom/2018-02-26
! genodelabs/src/decorator/2018-02-26
! genodelabs/src/floating_window_layouter/2018-02-26

In contrast to the list of 'used_apis' of a source archive, the content of
the 'archives' file denotes the origin of the respective archives
("genodelabs"), the archive type, followed by the versioned name of the
archive.

An 'archives' file may specify raw archives, source archives, or package
archives (as type 'pkg'). It thereby allows the expression of _runtime
dependencies_. If a package archive lists another package archive, it inherits
the content of the listed archive. This way, a new package archive may easily
customize an existing package archive.

A package archive does not specify binary archives directly as they differ
between the architecture and are already referenced by the source archives.

In addition to an 'archives' file, a package archive is expected to contain
a 'README' file explaining the purpose of the collection.


Depot structure
~~~~~~~~~~~~~~~

Archives are stored within a directory tree called _depot/_. The depot
is structured as follows:

! <user>/src/<name>/<version>/
! <user>/api/<name>/<version>/
! <user>/raw/<name>/<version>/
! <user>/pkg/<name>/<version>/
! <user>/bin/<arch>/<src-name>/<src-version>/

The <user> stands for the origin of the contained archives. For example, the
official archives provided by Genode Labs reside in a _genodelabs/_
subdirectory.
Subsuming archives in a subdirectory that correspond to their origin
(user) serves two purposes. First, it provides a user-local name space for
versioning archives. E.g., there might be two versions of a
'nitpicker/2017-04-15' source archive, one by "genodelabs" and one by
"nfeske". However, since each version resides in its origin's subdirectory,
version-naming conflicts between different origins cannot happen. Second, by
allowing multiple archive origins in the depot side-by-side, package archives
may incorporate archives of different origins, which fosters the goal of a
federalistic development, where contributions of different origins can be
easily combined.

The actual archives are stored in the subdirectories named after the archive
types ('raw', 'api', 'src', 'bin', 'pkg'). Archives contained in the _bin/_
subdirectories are further subdivided in the various architectures (like
'x86_64', or 'arm_v7a').


Depot management
~~~~~~~~~~~~~~~~

The tools for managing the depot content reside under the _tool/depot/_
directory. When invoked without arguments, each tool prints a brief
description of the tool and its arguments.

Unless stated otherwise, the tools are able to consume any number of archives
as arguments. By default, they perform their work sequentially. This can be
changed by the '-j<N>' argument, where <N> denotes the desired level of
parallelization. For example, by specifying '-j4' to the _tool/depot/build_
tool, four concurrent jobs are executed during the creation of binary archives.


Downloading archives
--------------------

The depot can be populated with archives in two ways, either by creating
the content from locally available source codes as explained by Section
[Automated extraction of archives from the source tree], or by downloading
ready-to-use archives from a web server.

In order to download archives originating from a specific user,
the download tool expects user-specific information to be defined
at _<repo>/sculpt/depot/<user>/_ where _<repo>_ can be any subdirectory
under _<genode-dir>/repos/_. For reference, the information for the
official "genodelabs" depot user is located at _gems/sculpt/depot/genodelabs/_.

! <user>/pubkey
! <user>/download

:_pubkey_: contains the public key of the GPG key pair used by the creator
  (aka "user") of the to-be-downloaded archives for signing the archives. The
  file contains the ASCII-armored version of the public key.

:_download_: contains the base URL of the web server where to fetch archives
  from. The web server is expected to mirror the structure of the depot.
  That is, the base URL is followed by a sub directory for the user,
  which contains the archive-type-specific subdirectories.

If both the public key and the download locations are defined, the download
tool can be used as follows:

! ./tool/depot/download genodelabs/src/zlib/2022-02-27

The tool automatically downloads the specified archives and their
dependencies. For example, as the zlib depends on the libc API, the libc API
archive is downloaded as well. All archive types are accepted as arguments
including binary and package archives. Furthermore, it is possible to download
all binary archives referenced by a package archive. For example, the
following command downloads the window-manager (wm) package archive, including
all binary archives, for the 64-bit x86 architecture. Downloaded binary
archives are always accompanied with their corresponding source and used API
archives.

! ./tool/depot/download genodelabs/pkg/x86_64/wm/2022-04-12

Archive content is not downloaded directly to the depot. Instead, the
individual archives and signature files are downloaded to a quarantine area in
the form of a _public/_ directory located in the root of Genode's source tree.
As its name suggests, the _public/_ directory contains data that is imported
from or to-be exported to the public. The download tool populates it with the
downloaded archives in their compressed form accompanied with their
signatures.

The compressed archives are not extracted before their signature is checked
against the public key defined at _depot/<user>/pubkey_. If however the
signature is valid, the archive content is imported to the target destination
within the depot. This procedure ensures that depot content - whenever
downloaded - is blessed by the cryptographic signature of its creator.


Building binary archives from source archives
---------------------------------------------

With the depot populated with source and API archives, one can use the
_tool/depot/build_ tool to produce binary archives. The arguments have the
form '<user>/bin/<arch>/<src-name>' where '<arch>' stands for the targeted
CPU architecture. For example, the following command builds the 'zlib'
library for the 64-bit x86 architecture. It executes four concurrent jobs
during the build process.

! ./tool/depot/build genodelabs/bin/x86_64/zlib/2022-02-27 -j4

Note that the command expects a specific version of the source archive as
argument. The depot may contain several versions. So the user has to decide,
which one to build.

After the tool is finished, the freshly built binary archive can be found in
the depot within the _genodelabs/bin/<arch>/<src>/<version>/_ subdirectory.
Only the final result of the built process is preserved. In the example above,
that would be the _zlib.lib.so_ library.

For debugging purposes, it might be interesting to inspect the intermediate
state of the build. This is possible by adding 'KEEP_BUILD_DIR=1' as argument
to the build command. The binary's intermediate build directory can be
found besides the binary archive's location named with a '.build' suffix.

By default, the build tool won't attempt to rebuild a binary archive that is
already present in the depot. However, it is possible to force a rebuild via
the 'REBUILD=1' argument.


Publishing archives
-------------------

Archives located in the depot can be conveniently made available to the public
using the _tool/depot/publish_ tool. Given an archive path, the tool takes
care of determining all archives that are implicitly needed by the specified
one, wrapping the archive's content into compressed tar archives, and signing
those.

As a precondition, the tool requires you to possess the private key that
matches the _sculpt/depot/<you>/pubkey_ file as found within one of the
available repositories. The key pair should be present in the key ring of your
GNU privacy guard.

To publish archives, one needs to provide the specific version to publish.
For example:

! ./tool/depot/publish <you>/pkg/x86_64/wm/2022-04-12

To accommodate the common case of publishing the current version of the source
tree, there exists the following shortcut:

! ./tool/depot/publish_current <you>/pkg/x86_64/wm

The command checks that the specified archive and all dependencies are present
in the depot. It then proceeds with the archiving and signing operations. For
the latter, the pass phrase for your private key will be requested. The
publish tool outputs the information about the processed archives, e.g.:

! publish /.../public/<you>/api/framebuffer_session/2020-06-28.tar.xz
! publish /.../public/<you>/api/gems/2022-02-14.tar.xz
! publish /.../public/<you>/api/gui_session/2020-06-28.tar.xz
! publish /.../public/<you>/api/input_session/2022-02-14.tar.xz
! publish /.../public/<you>/api/os/2022-04-12.tar.xz
! publish /.../public/<you>/api/report_session/2020-03-25.tar.xz
! publish /.../public/<you>/api/sandbox/2021-06-24.tar.xz
! publish /.../public/<you>/api/timer_session/2021-04-19.tar.xz
! publish /.../public/<you>/bin/x86_64/init/2022-04-12.tar.xz
! publish /.../public/<you>/bin/x86_64/report_rom/2022-04-12.tar.xz
! publish /.../public/<you>/bin/x86_64/wm/2022-04-12.tar.xz
! publish /.../public/<you>/pkg/wm/2022-04-12.tar.xz
! publish /.../public/<you>/raw/wm/2020-06-21.tar.xz
! publish /.../public/<you>/src/init/2022-04-12.tar.xz
! publish /.../public/<you>/src/report_rom/2022-04-12.tar.xz
! publish /.../public/<you>/src/wm/2022-04-12.tar.xz


According to the output, the tool populates a directory called _public/_
at the root of the Genode source tree with the to-be-published archives.
The content of the _public/_ directory is now ready to be copied to a
web server, e.g., by using rsync.


Automated extraction of archives from the source tree
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Genode users are expected to populate their local depot with content obtained
via the _tool/depot/download_ tool. However, Genode developers need a way to
create depot archives locally in order to make them available to users. Thanks
to the _tool/depot/extract_ tool, the assembly of archives does not need to be
a manual process. Instead, archives can be conveniently generated out of the
source codes present in the Genode source tree and the _contrib/_ directory.

However, the granularity of splitting source code into archives, the
definition of what a particular API entails, and the relationship between
archives must be augmented by the archive creator as this kind of information
is not present in the source tree as is. This is where so-called "archive
recipes" enter the picture. An archive recipe defines the content of an
archive. Such recipes can be located at an _recipes/_ subdirectory of any
source-code repository, similar to how port descriptions and run scripts
are organized. Each _recipe/_ directory contains subdirectories for the
archive types, which, in turn, contain a directory for each archive. The
latter is called a _recipe directory_.

Recipe directory
----------------

The recipe directory is named after the archive _omitting the archive version_
and contains at least one file named _hash_. This file defines the version
of the archive along with a hash value of the archive's content
separated by a space character. By tying the version name to a particular hash
value, the _extract_ tool is able to detect the appropriate points in time
whenever the version should be increased due to a change of the archive's
content.

API, source, and raw-data archive recipes
-----------------------------------------

Recipe directories for API, source, or raw-data archives contain a
_content.mk_ file that defines the archive's content in the form of make
rules. The content.mk file is executed from the archive's location within
the depot. Hence, the contained rules can refer to archive-relative files as
targets. The first (default) rule of the content.mk file is executed with a
customized make environment:

:GENODE_DIR: A variable that holds the path to the root of the Genode source tree,
:REP_DIR: A variable with the path to the source code repository where the recipe
  is located
:port_dir: A make function that returns the directory of a port within the
  _contrib/_ directory. The function expects the location of the
  corresponding port file as argument, for example, the 'zlib' recipe
  residing in the _libports/_ repository may specify '$(REP_DIR)/ports/zlib'
  to access the 3rd-party zlib source code.

Source archive recipes contain simplified versions of the 'used_apis' and
(for libraries) 'api' files as found in the archives. In contrast to the
depot's counterparts of these files, which contain version-suffixed names,
the files contained in recipe directories omit the version suffix. This
is possible because the extract tool always extracts the _current_ version
of a given archive from the source tree. This current version is already
defined in the corresponding recipe directory.

Package-archive recipes
-----------------------

The recipe directory for a package archive contains the verbatim content of
the to-be-created package archive except for the _archives_ file. All other
files are copied verbatim to the archive. The content of the recipe's
_archives_ file may omit the version information from the listed ingredients.
Furthermore, the user part of each entry can be left blank by using '_' as a
wildcard. When generating the package archive from the recipe, the extract
tool will replace this wildcard with the user that creates the archive.


Convenience front-end to the extract, build tools
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For developers, the work flow of interacting with the depot is most often the
combination of the _extract_ and _build_ tools whereas the latter expects
concrete version names as arguments. The _create_ tool accelerates this common
usage pattern by allowing the user to omit the version names. Operations
implicitly refer to the _current_ version of the archives as defined in
the recipes.

Furthermore, the _create_ tool is able to manage version updates for the
developer. If invoked with the argument 'UPDATE_VERSIONS=1', it automatically
updates hash files of the involved recipes by taking the current date as
version name. This is a valuable assistance in situations where a commonly
used API changes. In this case, the versions of the API and all dependent
archives must be increased, which would be a labour-intensive task otherwise.
If the depot already contains an archive of the current version, the create
tools won't re-create the depot archive by default. Local modifications of
the source code in the repository do not automatically result in a new archive.
To ensure that the depot archive is current, one can specify 'FORCE=1' when
executing the create tool. With this argument, existing depot archives are replaced by
freshly extracted ones and version updates are detected. When specified for
binary archives, 'FORCE=1' normally implies 'REBUILD=1'. To prevent
the superfluous rebuild of binary archives whose source versions remain
unchanged, 'FORCE=1' can be combined with the argument 'REBUILD='.


Accessing depot content from run scripts
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The depot tools are not meant to replace the run tool but rather to complement
it. When both tools are combined, the run tool implicitly refers to "current"
archive versions as defined for the archive's corresponding recipes. This way,
the regular run-tool work flow can be maintained while attaining a
productivity boost by fetching content from the depot instead of building it.

Run scripts can use the 'import_from_depot' function to incorporate archive
content from the depot into a scenario. It must be called after the
'create_boot_directory' function and takes any number of pkg, src, or raw
archives as arguments. An archive is specified as depot-relative path of the
form '<user>/<type>/name'. Run scripts may call 'import_from_depot'
repeatedly. Each argument can refer to a specific version of an archive or
just the version-less archive name. In the latter case, the current version
(as defined by a corresponding archive recipe in the source tree) is used.

If a 'src' archive is specified, the run tool integrates the content of
the corresponding binary archive into the scenario. The binary archives
are selected according the spec values as defined for the build directory.

The following excerpt of a run script incorporates the content of
several binary archives into a system scenario. The 'base_src' function
is provided by the run tool and returns the name of an archive with the
kernel-specific ingredients. It depends on the 'KERNEL' and 'BOARD'
definition in the build directory.

! import_from_depot [depot_user]/src/[base_src] \
!                   [depot_user]/src/report_rom \
!                   [depot_user]/src/fs_rom \
!                   [depot_user]/src/vfs \
!                   [depot_user]/src/init

The 'depot_user' function returns the name of the depot sub directory from
where the archives should be obtained. It returns "genodelabs" by default.
This default can be overridden via the '--depot-user' argument of the run tool.
For example, the following line in the _<build-dir>/etc/build.conf_ file
instructs the 'import_from_depot' call above to obtain the depot content from
_depot/test/_.

! RUN_OPT += --depot-user test


Automated depot management
--------------------------

When using the 'import_from_depot' mechanism of the run tool, one frequently
encounters a situation where the depot lacks a particular archive.
Whenever the run tool detects such a situation, it prompts the user to
manually curate the depot content via the _tool/depot/create_ tool.
The need for such manual steps negatively interferes with the development
workflow. The right manual steps are sometimes not straight-forward to find,
in particular after switching between Git branches.

To relieve the developer from this uncreative manual labor, the run tool
provides the option '--depot-auto-update' for managing the depot automatically
according to the needs of the executed run script. To enable this option, use
the following line in the build configuration:

! RUN_OPT += --depot-auto-update

If enabled, the run tool automatically invokes the right
depot-management commands to populate the depot with the required archives,
and to ensure the consistency of the depot content with the current version of
the source tree. The feature comes at the price of a delay when executing the
run script because the consistency check involves the extraction of all used
source archives from the source tree. In regular run scripts, this delay
is barely noticeable. Only when working with a run script of a large system,
it may be better to leave the depot auto update disabled.

Please note that the use of the automated depot update may result in version
updates of the corresponding depot recipes in the source tree (recipe hash
files). It is a good practice to review and commit those hash files once the
local changes in the source tree have reached a good shape.


Selectively overriding depot content
------------------------------------

While working on a component that is embedded in a complex system scenario,
the advantages of the run-tool's work flow and the depot can easily be
combined. The majority of the scenario's content may come from the depot via
the 'import_from_depot' mechanism. Because fetching content from the depot
sidesteps the build system for those components, the system integration step
becomes very quick. It is still possible to override selected components by
freshly built ones. For example, while working on the graphical _terminal_
component, one may combine the following lines in one run script:

! create_boot_directory
! ...
! import_from_depot genodelabs/pkg/terminal
! ...
! build { server/terminal }
! build_boot_image { terminal }

Since, the _pkg/terminal_ package is imported from the depot, the scenario
obtains all ingredients needed to spawn a graphical terminal such as font and
configuration data. The package also contains the 'terminal' binary. However,
as we want to use our freshly compiled binary instead, we override the
'terminal' with our customized version by specifying the binary name in the
'build_boot_image' step.

The same approach is convenient for instrumenting low-level parts of the
framework while debugging a larger scenario. As the low-level parts reside
within the dynamic linker, we can explicitly build the dynamic linker _lib/ld_
and integrate the resulting _ld.lib.so_ binary as boot module:

! create_boot_directory
! ...
! import_from_depot genodelabs/src/[base_src]
! ...
! build { lib/ld }
! build_boot_image { ld.lib.so }


Static code analysis
====================

The Clang static analyzer tool can analyze source code in C and C++ projects
to find bugs at compile time:

:Clang static analyzer:
  [https://clang-analyzer.llvm.org]

With this tool enabled, Genode users can check and ensure the quality of
Genode components. It can be invoked during make invocations and during
the creation of packages.

For the invocation of _make_ within a Genode build directory, the
STATIC_ANALYZE variable on the command line prompts the static analyzer to
run next to the actual build step.

! STATIC_ANALYZE=1 make -C build/x86_64 KERNEL=... run/...

For analyzing packages, the wrapper tool _tool/depot/static_analyze_
becomes handy. It can be combined with the _tool/depot/*_ tools to
take effect:

! tool/depot/static_analyze tool/depot/create <user>/pkg/...

The results of the static-analyzer tool are generated in the form of HTML
pages and can be inspected afterwards. The following example output showcases
a run of the static analyzer tool:

! make: Entering directory '../genode/build/x86_64'
! checking library dependencies...
! scan-build: Using '/usr/lib/llvm-6.0/bin/clang' for static analysis
! ...
!
! LINK     init
! scan-build: 0 bugs found.
! scan-build: The analyzer encountered problems on some source files.
! scan-build: Preprocessed versions of these sources were deposited in
!             '/tmp/scan-build-2018-11-28-111203-20081-1/failures'.

This feature is known to work well with Clang 6.0 on Ubuntu 16.04. The
steps to provide the required tools on Linux are like follows.

! sudo apt install clang-tools-6.0
! cd $HOME/bin
! ln -s $(which scan-build-6.0) scan-build


Git flow
========

The official Genode Git repository is available at the project's GitHub
site:

:GitHub project:

  https://github.com/genodelabs/genode


Master and staging
~~~~~~~~~~~~~~~~~~

The official Git repository has two branches "master" and "staging".


Master branch
-------------

The master branch is the recommended branch for users of the framework.
It is known to have passed quality tests. The existing history of this
branch is fixed and will never change.


Staging branch
--------------

The staging branch contains the commits that are scheduled for inclusion
into the master branch. However, before changes are merged into the master
branch, they are subjected to quality-assurance measures conducted by
Genode Labs. Those measures include the successful building of the framework
for all base platforms and the passing of automated tests. After changes
enter the staging branch, those quality-assurance measures are expected to
fail. If so, the changes are successively refined by a series of _fixup_
commits. Each fixup commit should refer to the commit it is refining using a
commit message as follows:

! fixup "<commit message of the refined commit>"

If the fixup is non-trivial, change the "fixup" prefix to "squash" and add
a more elaborative description to the commit message.

Once the staging branch passes the quality-assurance measures, the Genode
maintainers tidy-up the history of the staging branch by merging all fixup
commits with their respective original commit. The resulting commits are then
merged on top of the master branch and the staging branch is reset to the new
master branch.

Note that the staging branch is volatile. In contrast to the master branch,
its history is not stable. Hence, it should not be used to base developments
on.


Release version
---------------

The version number of a Genode release refers to the release date. The
two-digit major number corresponds to the last two digits of the year and
the two-digit minor number corresponds to the month. For example, "17.02".

Each Genode release represents a snapshot of the master branch taken at
release time. It is complemented by the following commits:

* "Release notes for version <version>" containing the release documentation
  in the form of a text file at _doc/release_notes_,
* "News item for Genode <version>" containing the release announcement as
  published at the _genode.org_ website,
* "Version: <version>" with the adaptation of the _VERSION_ file.

The latter commit is tagged with the version number. The tag is signed by one
of the mainline developers.


Development practice
~~~~~~~~~~~~~~~~~~~~

Each developer maintains a fork of Genode's Git repository. To facilitate
close collaboration with the developer community, it is recommended
to host the fork on GitHub. Open a GitHub account, use GitHub's web
interface to create a new fork, and follow the steps given by GitHub
to fetch the cloned repository to your development machine.

In the following, we refer to the official Genode repository as
"genodelabs/genode". To conveniently follow the project's mainline
development, it is recommended to register the official repository as a
"remote" in your Git repository:

! git remote add genodelabs https://github.com/genodelabs/genode.git

Once, the official repository is known to your clone, you can fetch new
official revisions via

! git fetch genodelabs


Topic branches
--------------

As a rule of thumb, every line of development has a corresponding
topic in the issue tracker. This is the place where the developers discuss and
review
the ongoing work. Hence, when starting a new line of development, the first
step should be the creation of a new topic.

:Issue tracker:

  [https://github.com/genodelabs/genode/issues]

The new topic should be accompanied with a short description about the
motivation behind the line of work and the taken approach.
The second step is the creation of a dedicated topic branch in the developer's
fork of Genode's Git repository.

! git checkout -b issue<number> genodelabs/master

The new topic branch should be based on the
most current _genodelabs/master_ branch. This eases the later integration of
the topic branch into the mainline development.

While working on a topic branch, it is recommended to commit many small
intermediate steps. This is useful to keep track of the line of thoughts
during development. This history is regarded as volatile. That is, it is not
set in stone. Hence, you as developer do not have to spend too much thoughts
on the commits during the actual development.

Once the work on the topic is completed and the topic branch is going to get
integrated into the mainline development, the developer curates the
topic-branch history so that a short and well-arranged sequence of commits
remains. This step is usually performed by interactively editing the
topic-branch history via the 'git rebase -i' command.
In many cases,
the entire topic branch can be squashed into a single commit. The goal behind
this curating step is to let the mainline history document the progress at a
level of detail that is meaningful for the users of the framework. The
mainline history should satisfy the following:

* The relationship of a commit with an issue at the issue tracker should be
  visible. For this reason, GitHub's annotations "Issue #n" and
  "Fixed #n" are added to the commit messages.

* Revisiting the history between Genode releases should clearly reveal the
  changes that potentially interest the users. I.e., when writing the
  quarterly release notes, the Genode developers go through the history and
  base the release-notes documentation on
  the information contained in the commit messages. This works best if each
  topic is comprised by a few commits with meaningful descriptions. This
  becomes hard if the history contains too many details.

* Each commit should represent a kind of "transaction" that can be reviewed
  independently without knowing too much context. This is hardly possible if
  intermediate steps that subsequently touch the same code are present as
  individual commits.

* It should be easy to selectively revert individual topics/features using git
  revert (e.g., when trouble-shooting). This is simple when each topic is
  represented by one or just a few commits.


Coding conventions
------------------

Genode's source code follows time-tested conventions regarding the
coding style and code pattern, which are important to follow. The coding style
is described in the following document:


:Coding-style Guidelines:

  [https://genode.org/documentation/developer-resources/coding_style]


Writing a commit message
------------------------

Commit messages should adhere the following convention.
The first line summarizes the commit using not more than 50 characters.
This line will be displayed by various tools. So it should express the basic
topic and eventually refer to an issue. For example:
! Add sanity checks in tool/tool_chain, fix #62

If the patch refers to an existing issue, add a reference to the
corresponding issue. If not, please consider opening an issue first. In the
case the patch is supposed to close an existing issue, add this information
using GitHub's conventions, e.g., by stating "Fix #45" in your commit
message, the issue will be closed automatically, by stating "Issue #45", the
commit will be displayed in the stream of discussion of the corresponding
issue.

After a blank line, a description of the patch follows. The description should
consider the following questions:
* Why is the patch needed?
* How does the patch achieve the goal?
* What are known consequences of this patch? Will it break API compatibility,
  or produce a follow-up issue?

Reconsider the documentation related to your patch: If the commit message
contains important information not present in the source code, this
information should better be placed into the code or the accompanied
documentation (e.g., in the form of a README file).


; XXX possible further topics
; * debugging
; * tracing
